Gradient Boost

Before moving forward with the to-do list, let’s throw a Random Forest to it.

Gradient boost

For many reasons, Random Forest is usually a very good baseline model. In this particular case I started with the polynomial OLS as baseline model, just because it was so evident from the correlations that the relationship between temperature and consumption follows a polynomial shape. But let’s go back to a beloved RF.

Model Cards provide a framework for transparent, responsible reporting. 
 Use the vetiver `.qmd` Quarto template as a place to start, 
 with vetiver.model_card()
Writing pin:
Name: 'wd-gb'
Version: 20250205T191659Z-5f397

<vetiver.vetiver_model.VetiverModel at 0x7fe5803fd1c0>

	Single Split		CV
	train	test	test	train
MAE - Mean Absolute Error	1.440284	2.022910	2.392068	1.265148
MSE - Mean Squared Error	4.068762	10.242401	24.012700	3.011794
RMSE - Root Mean Squared Error	2.017117	3.200375	4.067901	1.733948
R2 - Coefficient of Determination	0.961785	0.865192	-1.266718	0.969278
MAPE - Mean Absolute Percentage Error	0.129132	0.190952	0.258077	0.113032
EVS - Explained Variance Score	0.961785	0.872061	-0.439905	0.969278
MeAE - Median Absolute Error	1.042532	1.210331	1.568608	0.932174
D2 - D2 Absolute Error Score	0.806544	0.685701	-0.188265	0.822197
Pinball - Mean Pinball Loss	0.720142	1.011455	1.196034	0.632574

Scatter plot matrix

Test
Train

Observed vs. Predicted and Residuals vs. Predicted

Check for …

check the residuals to assess the goodness of fit.

white noise or is there a pattern?
heteroscedasticity?
non-linearity?

Test
Train

Normality of Residuals:

Check for …

Are residuals normally distributed?

Test
Train

Test
Train

Again, overfits a lot.

Parameter: param_model__learning_rate

Parameter: param_model__max_depth

Parameter: param_model__min_samples_leaf

Parameter: param_model__min_samples_split

Parameter: param_model__n_estimators

Parameter: param_model__subsample

Parameter: param_vars__columns

Best model

{'model__learning_rate': 0.1,
 'model__max_depth': 5,
 'model__min_samples_leaf': 5,
 'model__min_samples_split': 48,
 'model__n_estimators': 60,
 'model__subsample': 1,
 'vars__columns': ['rf_tu_mean', 'vp_std_mean']}

Pipeline(steps=[('vars', ColumnSelector(columns=['rf_tu_mean', 'vp_std_mean'])),
                ('model',
                 GradientBoostingRegressor(max_depth=5, min_samples_leaf=5,
                                           min_samples_split=48,
                                           n_estimators=60, random_state=7,
                                           subsample=1))])

In a Jupyter environment, please rerun this cell to show the HTML representation or trust the notebook.
On GitHub, the HTML representation is unable to render, please try loading this page with nbviewer.org.

Metrics

Plot
Table

	Single Split		CV
	train	test	test	train
MAE - Mean Absolute Error	1.649986	2.012082	2.399997	1.489268
MSE - Mean Squared Error	7.363867	10.039951	23.230062	4.818304
RMSE - Root Mean Squared Error	2.713645	3.168588	3.973658	2.194323
R2 - Coefficient of Determination	0.930837	0.867857	-1.367075	0.950921
MAPE - Mean Absolute Percentage Error	0.136069	0.196504	0.281227	0.123401
EVS - Explained Variance Score	0.930837	0.873994	-0.259795	0.950921
MeAE - Median Absolute Error	1.109518	1.195334	1.671029	0.994004
D2 - D2 Absolute Error Score	0.778377	0.687383	-0.290203	0.790657
Pinball - Mean Pinball Loss	0.824993	1.006041	1.199998	0.744634

Scatter plot matrix

Test
Train

Observed vs. Predicted and Residuals vs. Predicted

Check for …

check the residuals to assess the goodness of fit.

white noise or is there a pattern?
heteroscedasticity?
non-linearity?

Test
Train

Normality of Residuals:

Check for …

Are residuals normally distributed?

Test
Train

Test
Train

Leverage

Test
Train

Scale-Location plot

Test
Train

Residuals Autocorrelation Plot

Test
Train

Residuals vs Time

Test
Train

Compare vanilla vs. tuned

Metrics

Single split

Metrics based on the test set of the single split

Line plot
Radar plot

Cross validation

Line plot
Radar plot

Predictions, residuals, observed

Time vs. Residuals

Model details

5f397
8ef36

Pipeline(steps=[('vars',
                 ColumnSelector(columns=['tt_tu_mean', 'rf_tu_mean', 'td_mean',
                                         'vp_std_mean', 'tf_std_mean'])),
                ('model', GradientBoostingRegressor(random_state=7))])

Pipeline(steps=[('vars', ColumnSelector(columns=['rf_tu_mean', 'vp_std_mean'])),
                ('model',
                 GradientBoostingRegressor(max_depth=5, min_samples_leaf=5,
                                           min_samples_split=48,
                                           n_estimators=60, random_state=7,
                                           subsample=1))])

Gradient Boost

Gradient boost

Metrics

Scatter plot matrix

Observed vs. Predicted and Residuals vs. Predicted

Normality of Residuals:

Leverage

Scale-Location plot

Residuals Autocorrelation Plot

Residuals vs Time

Parameter: param_model__learning_rate

Parameter: param_model__max_depth

Parameter: param_model__min_samples_leaf

Parameter: param_model__min_samples_split

Parameter: param_model__n_estimators

Parameter: param_model__subsample

Parameter: param_vars__columns

Best model

Metrics

Scatter plot matrix

Observed vs. Predicted and Residuals vs. Predicted

Normality of Residuals:

Leverage

Scale-Location plot

Residuals Autocorrelation Plot

Residuals vs Time

Compare vanilla vs. tuned

Metrics

Single split

Cross validation

Predictions, residuals, observed

Time vs. Predicted and Observed

Time vs. Residuals

Model details

TODOs